Plan Explanations as Model Reconciliation: Moving Beyond Explanation as Soliloquy

The ability to explain the rationale behind a planner’s deliberative process is crucial to the realization of effective humanplanner interaction. However, in the context of human-inthe-loop planning, a significant challenge towards providing meaningful explanations arises due to the fact that the actor (planner) and the observer (human) are likely to have different models of the world, leading to a difference in the expected plan for the same perceived planning problem. In this paper, for the first time, we formalize this notion of MultiModel Planning (MMP) and describe how a planner can provide explanations of its plans in the context of such model differences. Specifically, we will pose the multi-model explanation generation problem as a model reconciliation problem and show how meaningful explanations may be affected by making corrections to the human model. We will also demonstrate the efficacy of our approach in randomly generated problems from benchmark planning domains, and motivate exciting avenues of future research in the MMP paradigm. With the continued progress of AI planning, the ability to execute plans in the real world has brought about the inevitability of interactions with humans in the environment.While traditional AI has considered humans as nothing but part of the environment in general, it has been argued (Kambhampati and Talamadupula 2014) that the presence of humans in the loop introduce unique challenges to the design of autonomy, especially in terms of how the decision process of an autonomous agent is perceived by humans. We postulate then a departure from traditional notions of automated planning in view of challenges of planning with humans in the loop. To the extent that such agents will be collaborating directly or at least share the same environment with humans, the design of autonomy cannot be completely agnostic of how its decisions or plans are perceived by the human, but rather must attempt to conform to human expectations. Indeed, a critical aspect of successful integration of autonomous agents into established human work-flows is how accepted standards of collaborative behavior can be used to inform the design of autonomy, so that the (actions of) artificial agents may be easily comprehended and hence perceived favorably from the point of view of the human. In the current climate of AI surrealism, and subsequent distrust/fear, the design of “explainable AI” is even more critical. Foremost among these challenges is developing automated agents that are able to provide explanations on their Figure 1: A human interacting with a planner (or a robot as an embodiment of it). The plans produced by the planner’s model M are evaluated by the human’s understanding M of the domain, which may be significantly different. Consequently, the optimal plans in the respective models π∗ H and π ∗ R will diverge, and the ability of humans to comprehend such plans is likely to be affected. In this paper we address the problem of explanation generation of plans considering such model differences under the umbrella of Multi-Model Planning or MMP. decision making and execution process (Langley 2016). The notion of explanations for planning problems has, in fact, been investigated before in (Kambhampati 1990) where different forms of explanations pertaining to a planning problem has been enumerated e.g. explanations of plan correctness and planning rationale. More recently, in (Sohrabi, Baier, and McIlraith 2011) authors have looked at how the behavior of an agent (available through observations) may be explained in terms of planning and plan recognition problems. However the underlying assumption in all these approaches is that the model used for explanations is the same as the model used for planning, and as such explanations can be given in terms of the given planning problem and the resulting plan generation process only. In this paper we argue that the core of this problem, especially in the context of human-in-the-loop planning, is the difference in understanding of the same planning problem between the human and the planner. We then formulate the task of plan explanations as a model reconciliation problem that can bring these models closer. Note that the human observer here is not trying to compare what he would have done for the planning problem, ar X iv :1 70 1. 08 31 7v 1 [ cs .A I] 2 8 Ja n 20 17 Figure 2: The Fetch in the crouched position with arm tucked (left), torso raised and arm outstretched (middle) and the rather tragic consequences of a mistaken action model (right). Unfortunately, this scenario happens to be a bit too close to the heart for the authors! :( but rather is trying to make sense of the plan produced by a planner given his understanding of the planning problem. Thus the above works on explanations over the plan generation process are still relevant, and we refer to the current setting as multi-model explanation generation. There are two ways in which such model differences might manifest themselves. • The planner as a student. Here the planner’s model is wrong in which case it should make an effort to bring it’s model closer to the ground truth. This problem has connections to model learning (Yang, Wu, and Jiang 2005; Zhuo, Nguyen, and Kambhampati 2013) and tracking (Bryce, Benton, and Boldt 2016), as well as works on explainable plan generation (Zhang et al. 2016; Zhang et al. 2015; Kulkarni et al. 2016). In general we can ask the question How can the planner bring its model closer to the perceived truth by changing its current model minimally? • The planner as a teacher. Here the planner has the ground truth and it can use it to correct a different system (including humans) using a faulty representation of a planning problem. An important use case here may be a robot explaining its actions to a non-expert, or a teacher agent explaining a task to a student agent. Again, we ask the question What can the planner provide as a model update so that it can effectively convey its ground truth? In general, the planner is likely to face a bit of both problems. In this paper we show how such model updates or explanations can be formulated concisely as the model reconciliation problem, which aims to bring two different planning problems closer to each other in the context of a plan that makes sense, i.e. is optimal, in one but not in the other, while making minimal corrections to one of the models. The FetchWorld domain We will now illustrate the concept of explanations via model reconciliation through an example in the Fetchworld domain (shown in Figure 2), based on the Fetch robot (Fetch Robotics 2016) whose design requires it to tuck its arms and lower its torso or crouch before moving something that is not obvious to a human who might be navigating it. This may lead to an unbalanced base and even toppling of the robot on its side if the human deems such actions as unnecessary. The move action for the robot is thus described in PDDL in the following model snippet (:action move :parameters (?from ?to location) :precondition (and (robot-at ?from) (hand-tucked) (crouched)) :effect (and (robot-at ?to) (not (robot-at ?from)))) (:action tuck :parameters () :precondition () :effect (and (hand-tucked) (crouched))) (:action crouch :parameters () :precondition () :effect (and (crouched))) Notice that the tuck action, provided by the manufacturer, also involves a lowering of torso so that the arm can rest on the base once it is tucked in. Now, consider a problem with the following initial and goal states (here, identical for both the robot and the human) (:init (:goal (block-at b1 loc1) (and (robot-at loc1) (block-at b1 loc2))) (hand-empty)) An optimal plan for the robot, in this case, involves a tuck action followed by a move (pick-up b1) (tuck) (move loc1 loc2) (put-down b1) The human, on the other hand, expects a much simpler model, as shown below. The move action does not have the preconditions for tucking the arm and lowering the torso, while tuck does not automatically lower the torso either.